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Background: Conjunction Assessment 


¢ Conjunction Assessment Risk Analysis (CARA) 


— Evaluates collision risk between two satellites expected to come in close 
proximity of each other (by calculating probability of collision [Pc]) 


— Mitigates collision risk, if necessary 


Contours of 
/ covariance ellipsoids 


Secondary 
object 


¢ Conjunctions usually identified several days before close approach 
— Risk usually follows more-or-less canonical development paradigm 

¢ However, sometimes risk increases or decreases quite suddenly 
— More insight needed into the circumstances behind such cases 
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Introduction 


¢ Tasked to analyze short notice events which are generally a result 
of unexpected, large state changes 
¢ Looked at all reported conjunctions for ca. 700 km protected 
missions from May 2015 though Feb 2016 
¢ Performed an analysis to determine whether there is any correlation 
between large state changes/late notice event identification and the 
following factors: 
— Sparse tracking 
— High drag objects 
— Space weather 
¢ Examined specific late notice events identified by missions to try to 
identify root cause 
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Broad Investigation of Large State Changes 


¢ Late-notice events usually driven by large changes in primary 
(protected) object or secondary object state 
¢ Main parameter to represent size of state change is component 
position difference divided by associated standard deviation (€/o) 
from covariance 
¢ Investigation determined actual frequency of large state changes, in 
both individual and combined states 
— Compared them to theoretically expected frequencies 
¢ Found that large changes (¢/o > 3) in individual object states occur 
much more frequently than theory dictates 
— Effect less pronounced in radial components and in events with Pc > 1e-5 
¢ Found combined state matched much closer to theoretical 
expectation, especially for radial and cross-track 


— |In-track is expected to be the most vulnerable to modeling errors, so not 
surprising that non-compliance largest in this component 
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Summary of “Other” Correlation Results 


¢ Pc correlation with large state changes in primary not very strong 
¢ Large state changes in the secondary do correlate to large changes 
in Pc, but not all that strongly 
— Value of Kendall’s Tau ranged from 0.37 to 0.6 
¢ Sparse tracking for secondary does not correlate with large state 
errors 


¢ Higher EDR values for secondary do not correlate with larger state 
errors 
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Correlations with Solar Activity 


¢ Elevated levels of solar activity can produce an unstable 
atmosphere whose density is difficult to model 
— More strongly true with geomagnetic storms (Dst, a,) 
—Can also be observed with EUV (F10, M10, S10, Y10, &c.) 
¢ Different possibilities for essence of the problem 
— Higher solar activity simpliciter 
— Mismatch between predicted and realized solar activity 
¢ Will investigate the former with correlation studies 
— Median F10 and a, over prediction interval 
— Peak a, over prediction interval 
¢ Will investigate the latter with case studies 
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Combined ¢/o vs Solar Indices: 
Tabular Summary 


Chi-Sq 
Median F10: Kendall 
All Data 


c/o>3 


: Kendall 
All Data 
c/o>3 


Peak Ap: Kendall 
All Data 
e/o>3 
e/o>5 


¢ Correlations are essentially nonexistent in all areas 
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Late-Notice HIE Case Studies 


¢ Examined four late-notice events that fell within data investigation 
period of current study 
—1 MAY 2015 to 1 FEB 2016 
¢ Events examined 
— Terra vs 38192, TCA 24 JUN 201 
— Aura vs 89477; TCA 29 AUG 2015 
— Terra vs 37131; TCA 19 DEC 2015 
—GPM vs 28685; TCA 5 SEP 2015 


Secondary Delta Position to Uncertainty 


* Determined not to be space weather related | == | 
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JSpOC Space Weather Information Files 


¢ JSpOC uses JBH09 
— JB08 + HASDM 
—Anemomilos DST prediction 
¢ Updated at JSpOC 3x per day 


¢ Model Input summary: 
— $10, S54 are daily and 54-day $10.7 index for >200 km heating of O by solar 
chromosphere 28.4-30.4 nm emissions in x10-22 Watts per meter squared per Hertz 


— M10, M54 are daily and 54-day M10.7 index for 100-110 km heating of O2 by solar 
photosphere 160 nm SRC emissions in x10-22 Watts per meter squared per Hertz 


— Y10, Y54 are daily and 54-day Y10.7 index for 85-90 km heating of N2, 02, H2O, NO by 
solar coronal 0.1-0.8 nm and Lya 121 nm emissions in x10-22 Watts per meter squared 
per Hertz 


— F10, F54 are daily and 54-day solar 10.7 cm radio flux in x10-22 Watts per meter 
squared per Hertz 


— a, is the 3-hour planetary geomagnetic 2 nT index (00-21 UT) 
— Dst is Disturbance Storm Time geomagnetic index in nT 
— DTC is delta exospheric temperature correction in units of K 
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Space Weather Evolution Charts 


* Upper left shows Dst; lower left shows a, 
¢ Black line is “issued” (definitive) data 
¢ Colored lines are predicted data 
— Each line begins when a given OD update executed 
— Each line shows predicted values of the geomagnetic index of choice 


¢ When Dst lines move to small positive value, prediction stops (zeroes in file) 
* When a, lines move to small negative value, prediction stops (ones in file) 


¢ Dst threshold for solar storm compensation engagement also shown 
¢ Upper right shows ¢/o for each component 

— Miss distance vs combined covariance 
¢ Lower right shows Pc vs time 
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Case Study #1: 
Terra vs 38192, TCA 24 JUN 2015 


ela vs Time to TCA 
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Space Weather Trade-Space Result: 
61 Hours to TCA 


¢ About half a day before spike in 
a,/Dst beg i ns 25994 vs_38192_1_20150624095651_20150621210556.swt 
— Some predicted increased Dst 
activity, but not of severity actually 
realized \ 
— Predictions at very end of storm 
over-predict Dst 
— Final prediction and shrinking 
covariance produces Pc drop off 
¢ SWTS indicates conjunction 
vulnerable to large Pc changes 
due to density mis-modeling 


Secondary Drag Scale Factor 


¢ Bottom line: missed solar storm 
and subsequent prediction 
failures produced late changes 


Primary Drag Scale Factor 
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Case Study #2: 
Aura vs 89477; TCA 29 AUG 2015 


Predicted and Definitive Dst History cla vs Time to TCA 
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¢ Run from update right as spike 
in a,/Dst is beginning 


—No predicted spike in relevant ASW 


Space weather file 


¢ Indicates that conjunction 
vulnerable to large Pc changes 
due to atmospheric mis- 
modeling 


¢ Bottom line: space weather 
predictions missed significant 
solar storm 


— Most likely cause of late-breaking 
change in Pc 


Omicron 


Secondary Drag Scale Factor 


Space Weather Trade-Space Result: 
Aura vs 89477; 56 Hours to TCA 


Effect of Atmospheric Density Changes on Pc 


56 hours to TCA 
28376_vs_89477_0_20150829075116_20150826234223.swt 


Primary Drag Scale Factor 
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Case Study #3: 


Terra vs 37131; TCA 19 DEC 2015 


elo vs Time to TCA 
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Space Weather Trade-Space Result: 
Terra vs 37131; 28 Hours to TCA 


¢ Run from update before 2 OOM 
change in Pc observed 
— Strange actual behavior in Dst 
— Modest unmodeled increase in Ap 
¢ SWTS indicates that conjunction ” 
vulnerable to Pc changes due to z 
atmospheric mis-modeling 


Effect of Atmospheric Density Changes on Pc 
28 hours to TCA 
25994 vs_37131_-8 20151220040553_20151219002506.swt 
= = es 


Secondary Drag Scale Factor 
| 


¢ Bottom line: odd space weather 
behavior, and deviation from 
predication, probably 
responsible for modest increase 
in Pc 


Primary Drag Scale Factor 
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Late-Breaking HIEs: 
Overall Summary 


¢ Large state changes occur more often than theory would indicate 
¢ Do not correlate at global level with any obvious causal condition 
— Light tracking, hard-to-maintain orbits, or generally elevated solar activity 
¢ Case studies indicate two culprits 
— Failure of JSpOC space weather predicted indices to predict solar storms 
— Edge cases for general screenings 
«Ils there any good news? 
— No, not really 
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Solar Storm Response — What are we doing? 


¢ CARA has begun receiving atmospheric model input data from 
JSpOC 
— Gives CARA analysts insight into what is being modeled 
—CARA analysts can work with outside experts (SWRC) to evaluate 
reasonableness and likelihood of predicted space weather events 
¢ CARA analysts can use model input information and outside 
evaluation of predictions to provided more nuanced feedback as to 
when to expect increased uncertainty and variation due to space 
weather 
— Additionally, as shown by this study, it is a great help for post-event analysis 
¢ Developing operational ConOps for how and when to apply space 
weather trade space with model insight 
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JSpOC Space Weather Information Files: 
Data Currency 


¢ Three types of data in file 
— “Issued” — definitive values for the solar/geomagnetic index, subjected to full 
availability of feeder data and consistency tests 
—“Nowcast” — initial observations of values, hand-scaled and not subject to 
consistency tests 
¢ Measurements stay in “nowcast” status for typically 24 hours 
— “Predicted” — values are predicted 
¢ EUV predicted values from 54- and sometimes 108-day autoregression analyses of 
past data 
¢ Geomagnetic indices are predicted from observed solar activity earlier in the solar 
rotation (and thus expected to become georelevant at a given future time) 
¢ Data type timing 
— Issued/Nowcast data used in propagating states from epoch to current time 
¢ Scaled/debiased with HASDM results 
— Predicted data used in propagating states from current time to TCA 
— Accuracy of predicted data can influence propagated result substantially 
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Normal Deviates and Chi-squared Variables 


¢ Let q and r be vectors of values that conform to a Gaussian 
distribution 
— These collection of values are called normal deviates 
¢ A normal deviate set can be transformed to a standard normal 
deviate by subtracting the mean and dividing by the standard 
deviation 
— This produces the so-called Z-variables 


¢ The sum of the squares of a series of standard normal deviates 
produces a chi-squared distribution, with the number of degrees of 
freedom equal to the number of series combined 


2 2 2 
Ze ee = X2dof 
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Normal Deviates in State Estimation 


¢ In a state estimate, the errors in each component (u, v, and w here) 
are expected to follow a Gaussian distribution 
— If all systematic errors have been solved for, only random error should remain 
¢ These errors can be standardized to the Z-formulation 


— Mean presumed to be zero (OD should produce unbiased results), so no need 
for explicit subtraction of mean 


¢ Sum of squares of these standardized errors should follow a chi- 
squared distribution with three degrees of freedom 


2 2 2 2 
Zi FL, +Zy, = X43dof 
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State Estimation Example Calculation 


¢ Let us presume we have a precision ephemeris, state estimate, and 


covariance about the state estimate 
— For the present, further presume covariance aligns perfectly with uvw frame 


(no off-diagonal terms) 
¢ Error vector e is position difference between state estimate and 


precision ephemeris, and covariance consists only of variances 


along the diagonal 
— Inverse of covariance matrix is straightforward 


: os 0 0 o> (0 0 
e=|e,|, C=|0 of 0 C'=| 0 Vo? O 
0 0 o« 0 0 Io 


¢ Resultant simple formula for chi-squared variables 


2 2 2 
é é 
-1 T Uu v u 2, 
| 9 ) Ai dor 


Vv w 


¢ Extension to case with off-diagonal terms straightforward 


= 
Omicron Pachura/Hejduk | Late-Notice HIEs | TBD SEP 2016 | 24 


Pearson Correlation Coefficient 


¢ Evaluates the degree of a linear relationship between two variables 


¢ Usually evaluated by the formula (s is sample standard deviation), 
with range of interesting and often not helpful outcomes 
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¢ Some interpretive guidance via relationship to r2 value from linear 
regression: square of Pearson = regression r2 
— Pearson value of 0.5 would equate to r° of 0.25—not very impressive 
¢ Really would like something that reveals even non-linear correlation 
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Kendall’s Tau 


¢ Rank correlation test 
— With two vectors of data X and Y, compares (Xi, Yi) to every other (Xj, Yj) 
— Pair is concordant if, when Xi>Xj, Yi>Yj; discordant if the opposite 
— Parameter is (# concordant pairs - # discordant pairs) / (total pairs) 
¢ So same range of values (-1 to 1) with same meaning 
¢ Much more robust test 
— Will find both linear and nonlinear correlation 
—Computationally expensive [~O(n2)], but computers are doing the work 
¢ Tied situations create problems 
—In present analysis, arises when comparing continuous to discrete distribution 


¢ e.g., €/o to tracking levels (because tracking levels are counting numbers, so can 
have multiple ¢/o values aligned with same tracking level) 


— Even more computationally expensive modifications to adjust for ties 
— Spot-checked these and saw no difference in computed result 
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Spearman’s Rho 


¢ Test of monotonicity, computed by summing squares of differences 
in rank 
— Mapped into same -1 to 1 range of values, with same interpretation 
¢ Computational formula 
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¢ Computationally easier but more vulnerable to outlier data 
¢ Usually larger than Kendall’s tau 
¢ Included here for consistency/contrast 
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Broad Investigation of Large State Changes 


¢ Determine actual frequency of large state changes, in both 
individual and combined states 
— Compare to theoretically expected frequencies 
¢ Determine whether broadly correlated with potential/expected 
causes 
— Low tracking 
— Harder-to-maintain orbits (larger energy dissipation rate) 
— General levels of solar activity (EUV and Joule atmospheric heating) 
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Large State Changes: 
Parameterization (1 of 3) 


¢ Main parameter to represent size of state change is component 
position difference divided by associated standard deviation (€/o) 

— Presumption of OD is that errors are normally distributed and unbiased 
—€ is difference in component position between subsequent state estimates 
— Oo is square root of associated variance from first state’s covariance 
— Dividing € by o creates standardized normal variable (u=0 because unbiased) 
— Set of these should thus conform to standard normal distribution 

¢ Same method currently used in CARA daily and HIE reports 


Omicron 


Delta Position to Uncertainty Ratio 
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Secondary Delta Position to Uncertainty 


Days to TCA 
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Large State Changes: 
Parameterization (2 of 3) 


¢ However... This is only true for the “diagonalized” situation, in 
which covariance axes and coordinate frame axes align 
— Results meaningful only if ellipse closely aligns with coordinate axes 
— Once ellipse rotated, then component errors are correlated 
¢ Individual component error distributions no longer independent random variables 
¢ How often are covariance error ellipsoids naturally diagonalized? 
— Not terrible assumption for individual satellites (primary, secondary) 
— More tenuous for combined situation (miss distance vs combined covariance) 
¢ Bottom line: ¢/o statistics at the component level must be used with 
care 
— When plotted against only positive axis, presume ¢€/o to be abs(e/o) 
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Large State Changes: 
Parameterization (3 of 3) 


¢ Comparison alternative: Mahalanobis distance 


— If individual component errors normally distributed, then sum of squares of 
individual ratios (€7/07) will constitute a 3-DoF y? distribution 


—Formulary eC-'e! properly considers all correlations and makes the calculation 
independent of coordinate system 


— Approach less frequently encountered, so less intuition built up around result 
— But will be supplied and examined along with Gaussian variables 
— Can also examine 2-DoF situation for only radial and in-track 

¢ More information on this later 
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Frequency of Large State Changes: 
Secondary Objects 
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Frequency of Large State Changes: 
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Summary of Frequencies: 
Primary and Secondary Objects 


¢ Data summary 
— Table below reports situation for which abs(e/o) > 3 
¢ Commonly-known theoretical “percentages” for univariate 
Gaussian distribution consider two-tailed results 
— 95.4% for 2-0 distribution considers results from 2.3% to 97.7% 
— 99.7% for 3-o distribution considers results from 0.15% to 99.85% 
¢ Actual percentages for primaries surprisingly large 


— Very similar for radial component; much larger differences with other two 


¢ Perhaps a little comfort in this, as radial generally most important component for CA 
Percent of Events in which abs(e/o) exceeds 3 
Primary Primary Primary 
Radial 1.57 1.33 0.37 0.30 0.09 0.08 0.08 0.06 
In-Track 5.88 3.31 1.00 0.41 0.26 0.16 0.23 
Cross-Track 13.53 7.10 2.64 0.88 0.34 0.13 0.22 


Omicron 
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Comparison of €/o to Theory: 
Primary and Secondary Objects 
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Comparison of €/o to Theory: 
Interpretation 


¢ Radial behaves reasonably well—better than theory until more 
extreme part of tails reached 
— Cannot see tail behavior very well in provided plots 
¢ In-track has non-theoretical distribution beyond about ¢/o > 1 
— As remarked previously, worse for secondaries than for primaries 
¢ Cross-track highly leptokurtic—peaked with very long tails 
— Does not match a Gaussian distribution at all 
¢ In using chi-squared distribution, 2-DoF framework gives more 
sanguine situation 
— Eliminates effect of large cross-track differences 
— Nonetheless, non-theory outliers dominate performance in the tails 
¢ None of these results sets match the theory particularly well 
¢ Immediate conclusion difficult 
—OD residuals suspected to be leptokurtic 
— Present trend could be extension of this 
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Combined Situation 


STATE-CHANGE FREQUENCY AND 
COMPARISON TO THEORY 
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Frequency of Large State Changes: 
Miss vs Combined Sigma 
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HASA ROBOTIC Cap, 


Comparison of €/o to Theory: 
Miss Component vs Combined Sigma 
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NASA ROBOTIC Capa 


Frequency of Large State Changes: 
Tabular Summary 


Percent of Events in which abs(€/o) exceeds 3 


Event Pc > 1E-05 > 1E-05 & APc> 100M > 1E-05 & APc >2 00M 


Secondary Secondary Secondary 
Radial 1.57 1.33 0.85 0.37 0.30 0.19 0.09 0.08 0.06 0.08 0.06 0.05 
In-Track 5.88 3.31 2.54 1.00 0.41 ( 0.26 0.16 0.23 0.15 
Cross-Track 13.53 7.10 0.90} 2.64 0.88] 0.10} 0.34 0.13 0.22 0.09 


¢ Values much closer to theoretical expectation, especially for radial 
and cross-track 


— |n-track is expected to be the most vulnerable to modeling errors, so not 
surprising that non-compliance largest in this component 


Omicron 
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Combined ¢/o vs Median F,,,: 
All Data 


In-Track efo vs F a 


Pearson=-0.0028185 Pearson=-0.011297 
Kendall=0.0081204 Kendall=-0.0096707 


Spearman=0.012195 : Spearman=-0.01425 
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NASA ROBOTIC CARA 
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Combined ¢/o vs Median F,,: 


Any Component abs(é/o) > 5 
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Issues in Comparison to Theory 


¢ Commonly-known “percentages” for univariate Gaussian 
distribution consider two-tailed results 
— 95.4% for 2-0 distribution considers results from 2.3% to 97.7% 
— 99.7% for 3-o distribution considers results from 0.15% to 99.85% 
¢ Potential double-counting of large state changes 
— Subsequent updates analyzed for large state change behavior 
—In achain of updates, return to normalcy will appear as a second large change 


— Demarcation between one and two events not so easy to define 
(S = small state change; L = large state change 
* SSLLSS-—one or two events? 
*SSSLSSLSS-—one or two events? 
*SSSSSSL —oneor two events (would it have been counted as two if one more 
update had been available? 
— For data-mining simplicity, all large changes counted, with the caveat that 
reported number might be twice as large as “actual” number 
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Solar Storm Predictions: 
What are we Doing? (1 of 2) 


¢ CARA member of NASA LWS space weather expert panel 
— Dr. Matt Hejduk as CA expert panel representative 
— Dr. Yihua Zheng as GSFC space physics representative, also representing 
mission interests 
¢ Purpose of panel to recommend NASA research investments to 
improve prediction and modeling 


— Will issue formal report of recommendations by December, as well as 
accompanying journal article 


— Will attempt to focus at least part of recommendation to address JSpOC 
situation 


¢ Hope to leverage report to push state of the art at JSpOC 


— However, from their perspective, a large investment was just made in 
atmospheric density prediction modeling; need to focus on other items 
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Solar Storm Predictions: 
What are we Doing? (2 of 2) 


¢ Will investigate whether file update frequency can be accelerated 
— Brief JSpOC on these results to show the problems that latencies create 
¢ See if there are mechanisms to improve efficiencies 
— Use SWTS function to determine whether such intervention is needed 


¢ Events that are not vulnerable to atmospheric density mismodeling would not require 
out-of-cycle updates 


— Would not have helped cases investigated here, as entire solar storms were 
missed 
¢ However, probably a fairly long time before there is much 
improvement with such scenarios 
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